Methods of communication for expandable telecommunication system

ABSTRACT

A means for transferring circuit switched data (CSD) and packet switched data (PSD) in an open, high speed, high bandwidth, expandable telecommunications system having a plurality of switching and non-switching nodes. The network may carry any type of information present in the system including voice, data, video, multimedia, control, configuration and maintenance, and the bandwidth of the network may be divided or shared across various information types. The network provides each node with essentially direct access to information (e.g., circuit switched data, packet switched data, etc.) originating from any port associated with any node served by the network. Different packet structures are provided for communicating circuit switched data, packet switched data, maintenance and control information, and the like. In one method, each node transmits one or more packets, each having an &#34;empty&#34; payload, which is received by other nodes that determine the source of the received packet and the packet&#39;s status. The receiving node inserts information (if any) it has for the transmitting node into the payload, after which it allows the packet to pass to the next node on the network. The packet traverses the complete network and returns with a &#34;full&#34; payload to the transmitting node, wherein the packet information is captured by the transmitting node. In this fashion, information of any type originating from any port served by any node may be transferred to any other port of the same or different node in the system. In another method, each node uses the network to transmit one or more packets, each of which has a &#34;full&#34; payload that contains information originating from the transmitting node. Each such packet is received by the other nodes, each of which determines the origin of the packet and whether any of the information contained therein is needed by the receiving node. If so, such information is captured from the payload before the packet passes to the next adjacent node. This process is repeated until each node on the network has transmitted one or more packets with a &#34;full&#34; payload and each such packet has traversed the complete network, thereby allowing each node access to the information originated by each other node. These two methods may be combined as well.

RELATED APPLICATIONS

This application is a divisional of Ser. No. 08/207,931, filed Mar. 8,1994, now U.S. Pat. No. 5,544,163.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates generally to the field oftelecommunications and, more specifically, to an architecture forconnecting a plurality of programmable telecommunications switches toprovide an expandable switching system and direct access for diversecommunications applications.

2. Discussion of the Prior Art

A fundamental consideration in any telecommunications system design isswitching capacity. Switching capacity must be analyzed in terms ofcurrent demand and projected demand in order to find a solution that iscost effective for both present and future service. For example, assumethat a developing country is in the process of building a basictelecommunications system and intends to provide service to most of itscurrent population. Such a population is most likely geographicallydistributed among small areas of high density (cities) and larger areasof low density (suburban and rural). In addition, the population isprobably growing, but at different rates in different areas. Thus, thechallenge for a telecommunications system designer is to providesufficient switching capacity to support satisfactory service to most orall of the population while also anticipating likely increases in futuredemand and providing for economical expansion.

Another example of the difficulty of providing appropriate switchingcapacity involves wireless or personal communications network (PCN)applications. These types of applications are based on micro-cellulararchitectures which require numerous base-stations, in close physicalproximity across a metropolitan area, with different switchingcapacities which aggregate to a large capacity.

A second fundamental consideration in telecommunications system designis providing for the addition of new features or services in the future.Telecommunications equipment and service continues to evolve rapidly,due in large part to the advent of digital technology. Even moredramatic advances are likely in the future, particularly as previouslyseparate industries such as cable television and local telephoneoperating companies integrate their services. Again, the challenge is tocreate a system which economically serves a present need, while alsoproviding flexible and inexpensive ways to integrate new features andservices as they become available.

Of the conventional approaches to the dual problems of providingadequate switching capacity along with access for new features andservices, most, if not all, suffer from one or both of two majordisadvantages: (1) there is insufficient bandwidth in the system tohandle information such as video or multimedia (in addition to voice anddata), (2) there is no direct, ready access to all of the informationpassing to or from the system, meaning there is no way to capture all ofthe information and distribute it to other switching systems orequipment, and (3) an increasingly large central switch is required toprovide access to some types of enhanced services.

One conventional approach may be referred to, for shorthand, as the "busextension" approach. In many conventional telecommunications switches,one or more internal buses are provided for carrying information,including voice, data and control information, between various parts ofthe switch. Buses are well suited for carrying such information since,by definition, multiple devices (e.g., circuit boards or cards) mayinterface with the buses and share them in accordance with a definedcommunication protocol. In a telecommunications switch, it is typical tofind one or more buses interconnecting a series of cards whichphysically terminate telephone lines or trunks with other cards whichperform switching, control or other functions.

As the shorthand name suggests, the concept underlying the bus extensionapproach is simply to connect additional cards, which provide additionalswitching capacity or other functions, with the existing buses. Inaddition to the two major disadvantages noted above, there are severalother disadvantages to this approach. First, there are physicallimitations as to the number of cards that can be physically connectedto or share the buses without degrading the system's performance.Second, in order to permit significant future expansion, the buses andother portions of the system must be constructed, in the first instance,to handle far greater traffic than is required prior to any expansion ofthe system. These limitations are related to the electrical andmechanical characteristics of the buses (or perhaps a particular one ofthe buses) and their effective operating speeds. Attempts to overcomethese limitations (e.g., using an excessively large number ofconnections to the bus) tends to increase the cost and complexity of the"base" or unexpanded system, possibly rendering the system too costlyfor some applications. There is also a limitation related to theprocessing power required to actually performing the switching functionsas well as control traffic on the buses.

Third, the bus structures found in many, if not most, conventionalswitching systems are generally designed solely for carrying out basiccall processing and switching functions and do not provide ready, directaccess to the ports for integrating new features and services.

Fourth, the bus structures are typically incapable of carrying packetswitched data or other types of information.

A second approach may be referred to as the "modular" approach forshorthand. In the modular approach, the concept is to provide aswitching system which is constructed from a series of essentiallyidentical modules. Each module provides a finite amount of switchingcapacity which may be added to an existing system (one or more at atime) to increase the overall capacity of the system.

Again, in addition to the major disadvantages noted earlier, the modularapproach has other deficiencies. In order to provide fully non-blockingoperation, each and every module as built must have the capability toreceive circuit switched data from every other module up to whatever themaximum number of modules may be. In terms of hardware, this means thateach module must be built with a sufficiently large memory to hold themaximum amount of circuit switched data which could be received if themaximum number of modules are connected together. For example, if eachmodule is capable of switching the equivalent of 64 ports and a maximumof eight modules may be connected together, then each module mustnecessarily contain a memory capable of holding circuit switched datafor (8×64)=512 ports. Thus, in the modular approach, it is the maximumswitching capacity of the fully expanded system which determines thesize of the memory that each module must have. For larger systems (i.e.,on the order of a few thousand ports or larger), constructing such amemory becomes impractical due to both the accompanying number ofphysical network/line interfaces as well as the additional circuitryneeded to control the memory.

Second, in order to maintain a truly "modular" system, it is impossibleto vary the switching capacity of individual modules.

Third, like the bus extension approach, the modular approach is orientedtoward performing basic switching operations and does not generallyoffer direct access to all the ports nor the capability of handlingpacket switched data or other types of information.

SUMMARY OF THE INVENTION

In brief summary, the present invention provides an open, high speed,high bandwidth digital communication network for connecting multipleprogrammable telecommunications switches to form a large capacity,non-blocking switching system. In a preferred embodiment, the network isimplemented using one or more rings which provide a medium fortransferring information over the network, and a plurality ofprogrammable switches, each of which appears as a node on the networkand serves a group of ports. Additional switches (nodes) may be added tothe network as desired to increase the system's switching capacity.

Each node includes circuitry for transmitting and receivingvariable-length, packetized information over the network, thus enablingeach node to receive information from or transmit information to allother nodes. The network may carry any type of information present inthe system including voice, data, video, multimedia, control,configuration and maintenance, and the bandwidth of the network may bedivided or shared across various information types.

In addition, devices or resources other than programmable switches mayalso act as nodes on the network, thereby gaining direct access to allinformation passing through the network. More specifically, voiceprocessing resources such as voice mail/message systems or otherenhanced services platforms may, by becoming nodes, gain direct accessto all ports served by the system without the need for a large centralswitch. The present invention's ability to transfer information of anytype, in a readily usable form, at high speed across the network enablesany service, feature or voice processing resource which is available ata given node to be provided to any port of the same or any other node.

The present invention also provides methods and packet structures forcommunicating information over the network. In general, different packetstructures are provided for communicating circuit switched information,voice processing information, data or maintenance information. However,all packets contain a control portion or header, which typicallyincludes address, status and other control information, and a payloadportion for carrying data. The combination of direct access to all portsand the ability to transfer information in packet form is highlycompatible with asynchronous transfer mode (ATM) operation on SONETnetworks.

In accordance with one method of transferring information between nodes,each node uses the network to transmit one or more packets, each ofwhich has an "empty" payload, which are received first by an adjacentnode. The adjacent node determines the source of the received packet andthe packet's status by the information contained in the control portionof the packet. If that adjacent node has information to send to the nodewhich transmitted the packet, the adjacent node inserts such informationinto the payload of the packet, then allows the packet to pass to thenext adjacent node on the network. If the adjacent node has noinformation for the node that originated the packet, the packet simplypasses to the next adjacent node on the network. This process isrepeated at each node until the packet traverses the complete networkand returns with a "full" payload to the node from which it originated.At that point, information which was inserted into the packet by othernodes is captured by the node which originated the packet. In turn, eachnode transmits an "empty" packet which traverses the network and returnswith information from other nodes. In this fashion, information of anytype originating from any port served by any node may be transferred toany other port of the same or different node in the system.

In accordance with an alternative method of transferring informationbetween nodes, each node uses the network to transmit one or morepackets, each of which has a "full" payload that contains informationoriginating from that node. Each such packet is initially received by anadjacent node which determines the origin of the packet and whether anyof the information contained therein is needed by that adjacent node. Ifso, such information is captured from the payload before the packetpasses to the next adjacent node. If no information is needed, thepacket simply passes to the next adjacent node. Again, this process isrepeated until each node on the network has transmitted one or morepackets with a "full" payload and each such packet has traversed thecomplete network, thereby allowing each node access to the informationoriginated by each other node.

By operating in accordance with either (or both) of the inventivemethods of transferring information, the capacity of each node totransfer information over the network may be advantageously establishedindependently from the other nodes. Further, a given node need onlycontain a memory which is sufficiently large to accommodate that node'sswitching (or voice processing) capacity and not the entire capacity ofthe system.

In another embodiment of the present invention, a second ring is used toconnect all of the nodes, thereby providing a second network. The secondnetwork effectively doubles the maximum switching capacity of the systemand also provides fault isolation in the event of a failure of the firstnetwork or one of the nodes.

In another embodiment of the present invention, one or more additionalnetworks are added to the nodes, further increasing the maximumswitching capacity of the system and providing redundancy.

In yet another embodiment of the present invention, one or more nodesmay be used to "bridge" one network to another. A bridge node is commonto two networks and is capable of exchanging information bidirectionallybetween such networks. A bridge node may also be used to connectnetworks which operate at different speeds.

BRIEF DESCRIPTION OF THE DRAWINGS

This invention is pointed out with particularity in the appended claims.The above and further advantages of this invention may be betterunderstood by referring to the following description taken inconjunction with the accompanying drawings, in which:

FIGS. 1A and 1B are a block diagram of an expandable telecommunicationssystem which employs a ring-type inter-nodal network to transferinformation between programmable switching nodes, all of which isconstructed in accordance with a preferred embodiment of the presentinvention;

FIGS. 1C and 1D are a block diagram of another embodiment of the presentinvention which employs a two-ring inter-nodal network to transferinformation between programmable switching nodes;

FIG. 1E shows various types of packets which may be used to transferinformation over the networks of FIGS. 1A through 1D;

FIG. 2A is a block diagram of a one type of programmable switching nodethat may be used in the systems of FIGS. 1A through 1D;

FIGS. 2B and 2C are a block diagram of a second type of programmableswitching node that may be used in the systems of FIGS. 1A through 1D;

FIGS. 3A, 3B, 3C, 3D and 3E are a block diagram of the nodal switchshown in FIGS. 2A through 2C are;

FIG. 3F is a detailed diagram of the transmitter and receiver memoriesshown in FIGS. 3B and 3C;

FIG. 4A is a block diagram which shows the receiving and transmittingfunctions involved in one method of transferring information over theinter-nodal networks of FIGS. 1A through 1D;

FIG. 4B is a flowchart showing the detailed steps of transferringcircuit switched information in accordance with the method depicted inFIG. 4A;

FIGS. 4C and 4D are a flowchart showing the detailed steps oftransferring both circuit switched data and packet switched data inaccordance with the method depicted in FIG. 4A;

FIG. 4E is a timing diagram showing the time relationships between nodesfor transferring both circuit switched data and packet switched data;

FIG. 5A is a block diagram which depicts a second method of transferringinformation over the inter-nodal networks of FIGS. 1A through 1D;

FIGS. 5B and 5C are a flowchart which depicts the detailed steps oftransferring both circuit switched data and packet switched data inaccordance with the method depicted in FIG. 5A;

FIGS. 6A and 6B are a block diagram of an expandable telecommunicationssystem which shows how communication may be maintained in event of afailure of one of the programmable switching nodes or a portion of theinter-nodal network;

FIG. 7 is a block diagram of another embodiment of the present inventionwhich employs two two-ring inter-nodal networks, one for redundancy, totransfer information between programmable switching nodes;

FIGS. 8A and 8B are a block diagram of another embodiment of the presentinvention which employs an inter-nodal network to transfer informationbetween one or more programmable switching nodes and one or more voiceprocessing resources nodes;

FIG. 8C is a block diagram of one of the voice processing resource nodesshown in FIGS. 8A and 8B;

FIGS. 9A and 9B are a block diagram of another embodiment of the presentinvention which employs a programmable switching node as a bridgebetween two inter-nodal networks;

FIG. 9C is a block diagram of the bridge node shown in FIGS. 9A and 9B;

FIG. 10A is a block diagram of another embodiment of the presentinvention which employs eight rings to transfer information betweenprogrammable switching nodes, demonstrating the further expandability ofthe switching system; and

FIGS. 10B and 10C are a block diagram of one of the switching nodes ofFIG. 10A.

DETAILED DESCRIPTION OF AN ILLUSTRATIVE EMBODIMENT

FIGS. 1A and 1B show a large capacity, expandable, fully programmabletelecommunications switching system 2. The system 2 includes a host 4and a series of programmable switching nodes 6a-6h. Each of nodes 6a-6hincludes a host interface which is connected in communicatingrelationship with host 4 by a local area network (LAN) such as Ethernetor by multiple asynchronous communication (RS-232) links 8. It should beunderstood that other types of host/node interfaces may be used insteadof or in addition to the LAN/RS-232 links 8. Although only a single host4 is shown, use of LAN 8 to provide host/node communications permitsmultiple hosts to control the system 2 (or parts thereof) by configuringeach host as a "client" and each node as a "server." For purposes ofimproved clarity in this drawing, the host interfaces of nodes 6a and6f-6h are truncated.

Each of nodes 6a-6h includes digital network/line interfaces forconnection with the public switched telephone network (PSTN) or aprivate network 10. The term "private network" is intended in a broadsense to refer to any network or line or other interface other than thePSTN. Again, for enhanced clarity, the network/line interfaces of nodes6b-6e are truncated. As shown by representative node 6g, thenetwork/line interfaces may terminate either digital networks or analogtrunks/lines, or combinations of both types. The network/line interfacesof a given node may include suitable interfaces for performingcommunications using ATM, Signalling System 7 (SS7), ISDN, T1/robbedbit, E1/CAS or other communication protocols.

Node 6g is nominally designated "master node A" (active master node) andnode 6h is nominally designated "master node B" (standby master node forredundancy). A synchronization reference line (ref 1 . . . ref n)extends from active master node 6g to each other switching node,although some such lines are truncated for clarity. As is explained indetail below in connection with FIGS. 3A through 3E, any of nodes 6a-6hmay be configured as the active master node or the standby master node.However, at any given time, there may be one and only one active masternode.

Nodes 6a-6h are connected together by an inter-nodal network 12 whichprovides for high speed, high bandwidth digital communications betweenthe nodes. As illustrated, inter-nodal network 12 may be implementedusing a ring which enables each of nodes 6a-6h to exchange packetizedinformation with each other node served by network 12. Inter-nodalnetwork 12 may also be implemented with any of a variety of other typesof communications networks, including Ethernet or other types of LANs,wireless communications networks and the PSTN (ATM/SONET). Using thePSTN for inter-nodal network 12 permits the nodes to be geographicallydistributed over large areas.

A general packet structure 14 for exchanging information over theinter-nodal network 12 consists of a control portion 16, a payloadportion 18 and a status and control portion 19. Details of variouspacket structures for transferring different types of information aredescribed below in connection with FIG. 1E.

Using inter-nodal network 12, a port of any given node may be connectedto any other port of the same node or any other node in a fullynon-blocking manner. In this preferred embodiment, with a total of eightswitching nodes 6a-6h interconnected by the inter-nodal network 12, ifall of the bandwidth of the inter-nodal network 12 is used fortransferring circuit switched data, the system 2 is capable of switching(8×2,048)=16,384 ports, which equates to 8,192 simultaneous, two-waycalls.

It should be understood that each of nodes 6a-6h operates independentlywith respect to the network/line interfaces terminated thereon. That is,any node may be removed or added to inter-nodal network 12 withoutimpairing the operations or network/line interfaces of the other nodes.Further, the switching capacity of each switching node may beestablished independently from the switching capacities of other nodes(i.e., "small" switches may be combined with "large" switches on thesame inter-nodal network 12). Thus, the overall switching capacity ofthe system 2 may be increased simply by adding additional switchingnodes to the inter-nodal network 12, subject to certain limitationsregarding the data transmission rate of that network, or additionalinter-nodal networks 12 which are discussed below.

The overall operation of system 2 is controlled by host 4, which iscommonly implemented with a personal computer (PC), workstation, faulttolerant or other computer on which a user's application software runs.Host 4 and each of nodes 6a-6h exchange messages over LAN/RS-232 links8. Such messages are typically used to configure the nodes as well asdirect call processing functions such as making connections andproviding communications services (i.e., tone detection, tone generationand conferencing).

FIGS. 1C and 1D show an expandable telecommunications switching system17 which is similar to system 2 of FIGS. 1A and 1B, except that tworings are used to form the inter-nodal network 12 which connects nodes6a-6h. PSTN/private network 10 is omitted for clarity. Throughout theremaining figures, the same reference numbers will be used to designatesimilar components or steps. Conceptually, each of the two rings may beconsidered a separate inter-nodal network (or, alternatively, may beconsidered separate channels within a single network) since informationmay be transferred between nodes using either ring independently fromthe other, thereby effectively doubling the maximum switching capacityas compared to that of system 2. Also, use of two rings provides faultisolation for the system 17. That is, should one ring fail (which wouldrender the entire, single ring system 2 inoperable), the second ring maycontinue to transfer information between nodes, thereby keeping thesystem 17 at least partially operational.

FIG. 1E shows preferred embodiments for several packets which may beused to transfer information over inter-nodal network 12. A circuitswitched data packet 3 and a voice processing packet 5 are similarlyconstructed and each includes a control portion which contains a busyindicator (BI) followed by address and control information. The busyindicator may be used, as described in detail below, to denote thecurrent status of a given packet as either "busy" (meaning the packetmay not be used by a node to transfer information) or "free".

The address information preferably includes an address for either thesource (SRC) node from which the packet originates or the destination(DEST) node for which the packet is intended, or both. Each address(source or destination) preferably includes a "network address" whichuniquely identifies a particular inter-nodal network. Suchidentification is necessary since, as described below, multipleinter-nodal networks may be used to connect the same or different groupsof nodes. Each address (source or destination) preferably also includesa "nodal address" which uniquely identifies a particular node on aparticular inter-nodal network. Additional address information mayinclude an explicit "port address" for uniquely identifying a particularport or groups of ports.

In general, packets 3 and 5, which carry circuit switched data, require"port addresses" since such data is subject to distribution acrossmultiple nodes and/or ports. As an alternative to explicit "portaddresses" (which, in the context of a large switching system wouldrepresent thousands of bytes of additional information carried by thepacket), implicit "port addresses" may be determined by maintaining apredetermined order of the circuit switched data within the payload. Forexample, packets 3 and 5 are depicted as having sufficient payloadcapacities to carry a total of 2,048 bytes of circuit switched data.When such bytes are placed in the payload, they are preferably arrangedin an order which corresponds exactly with the sequence of time slots ata given node. Specifically, the byte of circuit switched data whichcorresponds to the "first" time slot (time slot (TS) 0) of a given nodeis placed first in the payload, followed by the remaining bytes insequential order. By this arrangement, any given node may either loadcircuit switched data into or extract data from the payload and, bysimply counting the position of a particular byte relative to the firstbyte in the payload, know exactly the time slot with which the bytecorresponds.

In contrast, packets 7 and 9 do not generally require "port addresses"since the information carried by those types of packets is not circuitswitched data.

Additional information may be included in control portion 16 to specifythe packet type, the length of the packet, a packet sequence number orother information.

The length or payload capacity of each packet type may be varieddepending upon which node transmits a given packet. For example, thepayload capacities of packets 3 and 5 may be different so long as theyprovide sufficient capacity to carry circuit switched data up to themaximum number of ports switched or processed by a given node. Thus, ifa particular node is capable of switching or processing a maximum of2,048 ports, then that node preferably transmits packets 3 and 5 withpayloads having capacity for up to 2,048 bytes of circuit switched data.Similarly, if a different node is capable of switching only 512 ports,that node preferably transmits packets 3 and 5 with payloads havingcapacity for up to 512 bytes of circuit switched data.

The payload portions of all packet types are preferably followed bystatus and control information, which may include a checksum or otherinformation for error detection and correction.

A packet switched data packet 7 and a maintenance packet 9 are similarlyconstructed (their lengths or payload capacities are variable), exceptthat these types of packets do not carry circuit switched data but, asdescribed below, are intended to transfer packet switched data whichoriginates from a single point (source) and is destined to betransferred to another single point (destination) or to multiple singlepoints ("broadcast"). The status and control portions of packets 7 and 9may include information which indicates whether a destination node for agiven packet was able to accept the packet or was busy at the time ofreceipt and unable to accept the packet.

FIG. 2A shows the major functional components of a preferred embodimentof one type of programmable switching node which may be used in thesystems of FIGS. 1A through 1D. Digital or analog network/lineinterfaces are terminated on a series of line card input/output (IO)cards 20. In a preferred embodiment, network/line interfacesrepresenting a total of 2,048 ports may be terminated by line card IOcards 20. If desired, a redundant line card IO card 22 and redundant IObus 24 may optionally be provided to permit continued operation of theswitching node in the event of a failure of one of line card IO cards20.

A series of digital network T1, E1, J1 or analog trunk/line line cards26 communicate with line card IO cards 20 over line card (LC) IO lines28. Line cards 26 are also interfaced with redundant switching buses 30aand 30b. Again, if desired, an optional redundant line card 32 may beprovided, which communicates with redundant line card IO card 22 overredundant LC IO lines 34. Other types of network/line interfaces such asDS3, SONET or others may also be provided.

Diverse communications services such as tone detection and generation,conferencing, voice recorded announcements, call progress analysis,speech recognition, ADPCM compression and many others are provided byone or moremultifunction digital signal processing (MFDSP) cards 36.ISDN Primary Rate service and other packet communications services areprovided by one or more ISDN-24 cards 38. Redundant MFDSP cards 36 andredundant ISDN-24 cards 38 may be optionally included. Details of thepreferred construction and operation of MFDSP cards 36 and ISDN-24 cards38, as well as buses 30a and 30b, are disclosed in a co-pendingapplication, filed Jan. 5, 1993, Ser. No. 08/001,113, now issued as U.S.Pat. No. 5,349,579 and assigned to the assignee of the presentapplication and hereby incorporated by reference. Assuming it isequipped with one or more cards 36 or 38, a particular node may operateindependently from other nodes in terms of performing diversecommunications services. Alternatively, as described below, only onenode (or a subset of all of the nodes) may be equipped with cards 36 or38, and inter-nodal network 12 may be used to provide communicationsservices to other nodes which are not so equipped.

A ring (network) IO card 40a serves as an interface between one pair ofrings (designated Set A, Rings 1 and 2), which together are designatedinter-nodal network 12a, and a nodal switch 44a that is designated the"local bus master," the significance of which is described below. Afirst host interface 42a handles all communication between host 4 andthe node of FIG. 2A.

A second, redundant ring (network) IO card 40b serves as an interfacebetween a redundant pair of rings (designated Set B, Rings 3 and 4)which together form a second inter-nodal network 12b, and a redundantnodal switch 44b, which is preferably of the same construction as nodalswitch 44a. A second host interface 42b provides a communication linkwith host 4. A link 46 provides for communication between nodal switches44a and 44b. Link 46 is used only to connect a nodal switch which isoperating as the local bus master with another nodal switch which isoperating as a redundant local bus master.

In a preferred embodiment, line cards 26 perform real time callprocessing functions which are required by network/line interfaces,including analog to digital conversion, if necessary. Line cards 26transmit and receive time division multiplex (TDM) circuit switched dataover switching buses 30a and 30b. Each of nodal switches 44a and 44b,MFDSP cards 36 and ISDN-24 cards 38 receive, over the buses 30a and 30b,circuit switched data transmitted in all time slots from all line cards26. Each of nodal switches 44a and 44b, MFDSP cards 36 and ISDN-24 cards38 has the ability, under the direction of the local bus master (i.e.,nodal switch 44a), to transmit circuit switched data to the line cards26, over buses 30a and 30b, during predetermined time slots. Inaddition, switching buses 30a and 30b each include a high level datalink control (HDLC) bus over which CPUs in nodal switches 44a and 44b,MFDSP cards 36 and ISDN-24 cards 38 exchange control messages.

For convenience, throughout the remainder of this description, the term"local port" shall be used to refer, with respect to a given node, to atime slot containing circuit switched data transmitted from a line card26 to all nodal switches 44, MFDSP cards 36 and ISDN-24 cards 38 (ifany), or a time slot containing data transmitted from any nodal switch44, MFDSP card 36 or ISDN-24 card 38 to a line card 26. The term "remoteport" shall be used to refer, with respect to a given node, to a localport of a different node.

In a preferred embodiment, each node 6a-6h is capable of time switchingup to 2,048 local ports. Thus, in this preferred embodiment, each ofnodal switches 44a and 44b includes a time switch capable of switching2,048 time slots. In accordance with one aspect of the presentinvention, the switching memory of each nodal switch 44a and 44b needonly be sufficiently large to accommodate the maximum number of localports and not the switching capacity of the entire system. A significantadvantage of this aspect of the present invention may be appreciated bymomentarily referring again to FIGS. 1A and 1B. As mentioned above, apreferred embodiment of the system 2 is capable of switching a total of16,384 ports. However, the switch (nodal switch 44a) within each ofnodes 6a-6h need only contain a switching memory which is large enoughto switch 2,048 local ports, not 16,384 ports of the entire system 2. Asdescribed more fully below, it is the novel arrangement of inter-nodalnetwork 12 and its ability to transfer circuit switched data from onenode to any other node which provides, in effect, a second stage ofswitching which yields the high overall capacity of system 2.

FIGS. 2B and 2C show a preferred embodiment of a second type ofprogrammable switching node. This type of node is preferably based on anoff-the-shelf PC which includes a PC-486 (or equivalent) and peripherals48, an ISA (AT) bus 50 and a mass storage device 52. The PC-486 48 maybe used to run a user's application software and effectively operate asa host 4. Alternatively, an optional host interface 42a may be used toconnect an "external" host (such as host 4 in FIGS. 1A through 1D) tocontrol the node. In addition to components already identified inconnection with the preceding figure, several additional components areprovided in this embodiment. A voice processing resources bus interface54 provides bidirectional communication between switching bus 30a andtwo voice processing buses, PEB bus 60 and/or MVIP bus 62. PEB bus 60and MVIP bus 62 represent well known, "standard" interfaces forcommunicating with commercially available, widely used voice processingresources 56 and 58, respectively. For example, Dialogic Corporation ofNew Jersey produces a family of voice processing resource boards orcards which plug directly into PEB bus 60 and may be used in diverseapplications including voice mail, fax mail, interactive voice responseand others.

The detailed construction of a preferred embodiment of nodal switch 44ais shown in FIGS. 3A through 3E. A central processing unit (CPU) withassociated RAM/ROM 64 is connected in communicating relationship with aCPU address bus 114 and a CPU data bus 116. CPU 64 is also connected incommunicating relationship with an HDLC bus (part of switching buses 30aand 30b) and may, depending upon the configuration of nodal switch 44adiscussed below, also be connected in communicating relationship withhost 4.

A data transmitter 66 is connected in communicating relationship withCPU address and data buses 114 and 116 and two packet handling circuits78a and 78b. Transmitter 66 is also connected to receive circuitswitched data for local ports over switching bus 30a (redundantswitching bus 30b is omitted for clarity). As explained below, dependingupon its mode of operation, transmitter 66 may receive and time switchcircuit switched data which is flowing in a direction from a Line cardto a Switch (LSDATA) or, alternatively, may receive and time switch datawhich is flowing in a direction from a Switch to a Line card (SLDATA).Transmitter 66 includes two ring maps 96, 98, corresponding to rings 1and 2, respectively, a local sequential counter/map 100 and a quad-portlocal transmitter memory 102.

A data receiver 68 is connected in communicating relationship with CPUdata and address buses 114 and 116, and with a space switch controlcircuit 112 whose output is transmitted over switching bus 30a. Receiver68 may, in conjunction with space switch control circuit 112, dependingupon its mode of operation, output circuit switched data which flows ineither the SLDATA or LSDATA direction (e.g., whichever is opposite tothat of the data input to transmitter 66). Receiver 68 includes asequential count/map 104, a local time slot map 106, a tri-port localreceiver memory 108, a pad lookup memory 110, a dual-port local datapacket receiver memory 118 and a sequential map/control 120.

A high speed data receiver 70a is physically interfaced with ring 1 forreceiving information in packet form from that ring. Receiver 70a ispreferably implemented with a Hewlett-Packard Company HDMP-1014 receiverchip, which is an emitter coupled logic (ECL) device. Conversion circuit72a is connected to receive the output signals of receiver 70a andproduce output signals that are compatible with transistor-transistorlogic (TTL). The output of conversion circuit 72a is applied to amultiplexer 74a, which converts 16 bit data received from receiver 70ato 32 bit format. The output of multiplexer 74a is applied to afirst-in-first-out (FIFO) memory 76a, a packet control circuit 92a and aring select circuit 94. A transmit flag (XF) circuit 90a is connected topacket control circuit 92a. The output of FIFO 76a is connected topacket handling circuit 78a. A demultiplexer circuit 80a, conversioncircuit 82a and high speed data transmitter 84a perform functions whichare the complements of multiplexer 74a, conversion circuit 72a and datareceiver 70a, respectively. Transmitter 84a is preferably implementedwith a Hewlett-Packard Company HDMP-1012 transmitter chip.

Separate, but identical, circuitry is provided for interfacing with andtransferring information to or from ring 2. Like reference numbers areused to identify corresponding components. As explained below inconnection with FIGS. 6A and 6B, during periods of time when nodalswitch 44a operates in a "loopback" mode, the output of transmitter 84bis effectively connected to the input of receiver 70a, as indicated inphantom and reference number 71a. Similarly, the input of receiver 70bis effectively connected to the output of transmitter 84a, as indicatedby reference number 71b.

Nodal switch 44a includes additional components for timing andsynchronization functions, which are grouped together as master nodeoptions 65 and local bus master options 71. Master node options 65include an inter-nodal synchronization circuit 67 and a master ringoscillator 69. Synchronization circuit 67 generates reference signalsref 1 . . . ref n, each of which is supplied to one other switching node(see FIGS. 1A through 1D). Synchronization circuit 67 also generates anodal frame synchronization signal and a master ring clock signal, bothof which are supplied to the packet control circuits 92a and 92b.

Local bus master options 71 include a local bus HDLC control 73 and alocal synchronization circuit 75. Local bus HDLC control 73 is connectedin communicating relationship with CPU address and data buses 114 and116, respectively, and generates a series of control signals 1 . . . . nwhich are supplied to all other cards (i.e., other nodal switches, linecards, MFDSP cards and ISDN-24 cards) associated with a given node forcontrolling access to the HDLC bus.

Local synchronization circuit 75 receives two input signals. One inputsignal is either one of the ref 1 . . . ref n signals (if another nodalswitch is configured as the master node) or a loop timing source (if thenodal switch of FIGS. 3A through 3D is itself configured as the masternode). The frame synchronization signal to circuit 75 is obtained fromeither inter-nodal network (ring) 12 or one of ref 1 . . . ref n signals(if another nodal switch is configured as the master node). Circuit 75will self-generate the frame synchronization signal if it is itselfconfigured as the master node.

Further details regarding the construction of receiver memory 108 andtransmitter memory 102 are shown in FIG. 3F. Transmitter memory 102 isorganized into dual circuit switched data banks 122 and 126, and dualconstant areas 124 and 128. Similarly, receiver memory 108 is organizedinto dual circuit switched data banks 130 and 134, and dual constantareas 132 and 136. The dual circuit switched data banks of each memoryare operable, in conjunction with their respective maps and counters, totime switch circuit switched data. That is, during a given time slot, abyte of circuit switched data is written sequentially into a memorylocation in one of the circuit switched data banks, while circuitswitched data stored in the other circuit switched data bank is read"selectively." The term "selectively" is used in this description torefer a process of applying addresses which are supplied by a map.During alternate 125 μs time periods, the roles of the circuit switcheddata banks reverse, thus interchanging the time slots to effect timeswitching.

The constant areas of each memory are generally available for storage ofpacket switched data by CPU 64, although the CPU 64 may access anylocation in either memory.

Configuration, Synchronization and Initialization

Before proceeding with an overview of the operation of nodal switch 44a,it is helpful to understand how each switch may be configured to operateand what its responsibilities are in terms of system synchronization andinitialization. With reference again to FIGS. 1A, 1B and 3A through 3E,it should be understood that each programmable switching node 6a-6h mustcontain at least one, but may contain more than one, nodal switch 44a.It should also be understood that, in general, two types ofsynchronization must be considered: inter-nodal network synchronizationand PSTN (or private network) synchronization.

Each nodal switch 44a is preferably configurable, by software, tooperate as (1) a combination master node and local bus master, (2) alocal bus master only, or (3) neither a master node nor a local busmaster, but simply a "standard" switch. The configuration rules are asfollows. For each inter-nodal network 12, there must at any given timebe one and only nodal switch which is operating as the master node.Whichever nodal switch is operating as the master node may also operateas the local bus master for its node. Within a given node, there must atany given time be one and only one nodal switch which is operating asthe local bus master for that node. Lastly, within a given node, at anygiven time there may be one or more nodal switches operating as standardswitches.

The responsibilities of a nodal switch operating as the master node are:(1) interface to PSTN for loop timing source (via circuit 75) for bitsynchronization to digital networks of the PSTN; (2) generate systemwide maintenance packets which all other nodes use for framesynchronization to digital networks of PSTN (based upon the nodal framesynchronization signal generated by circuit 67); (3) generate aswitching reference clocking source (ref 1 . . . ref n) for bitsynchronization of all non-master nodes; (4) optionally transmit amaster framing signal over ref 1 . . . ref n; (5) generate a masterclock for the inter-nodal network (master ring clock); (6) break thenetwork (ring) clocking; and (7) keep the integrity of the inter-nodalnetwork intact.

The responsibilities of a nodal switch operating as a local bus masterare: (1) interface to PSTN loop timing source or ref 1 . . . ref n frommaster node for bit synchronization to digital networks of the PSTN; (2)accept system wide maintenance packets generated by the master node forframe synchronization to digital networks of the PSTN; (3) communicatewith the host; (4) communicate with all other cards in the node (othernodal switches, line cards, MFDSP cards and ISDN-24 cards) over the HDLCbus (controlled by controls signals 1 . . . n from HDLC control 73); and(5) generate nodal clock and flaming for all other cards in the node(local bus dock and local bus frame synchronization signals from circuit75).

The responsibilities of a nodal switch operating as a standard switchare: accept local bus clock and local bus frame synchronization signalsfrom local bus master.

The master node is responsible for initializing and configuring thesystem, which involves verifying the integrity and operability of theinter-nodal network 12 and, optionally, either assigning a nodal addressto each node or polling the nodes to determine their previously assignedaddresses. Once a node's address is assigned or determined, the masternode may interrogate that node (i.e., using maintenance packets overinter-nodal network 12) to obtain configuration information such asnodal type, types of PSTN interfaces and/or protocols, switchingcapacity or other information. The master node may also haveresponsibilities for performing maintenance and administrationfunctions. In addition, if multiple rings are used to implement anyinter-nodal network, the master node may assign each nodal switch aparticular ring for transmitting and receiving packets.

Overview of Operation

With reference to FIGS. 1C, 1D and 3A through 3F, an overview of theoperation of the system 17 will now be presented. Consideration will begiven first to how circuit switched data is handled. For purposes ofthis overview, it is assumed that system 17 is already initialized.

The LSDATA (or SLDATA) which is input to transmitter memory 102represents bytes of circuit switched data for local ports served by agiven node. These bytes are written sequentially into the circuitswitched data banks 122 and 126. Accordingly, the capacities of thosedata banks effectively determine the maximum number time slots which canbe time switched by nodal switch 44a. For purposes of this overview, itis assumed that each data bank has a capacity of 2,048 bytes, meaningthat a maximum of 2,048 local ports can be time switched by transmittermemory 102.

In order to make this "local" circuit switched data (stored in memory102) available to every other node served by inter-nodal network 12, oneof two methods may be used. In the first method, transmitter 66 andpacket handling circuit 78a (it is assumed that ring 1 is the ringassigned to this node for transmission of packets) formulate a packetwhose payload is "empty" (meaning that the payload contains no circuitswitched data, except for data from local ports which are connected toother local ports), but which has sufficient capacity to hold up to2,048 bytes of circuit switched data. Transmitter 84a then transmits the"empty" packet. If we assume, for example, that the "empty" packet istransmitted by node 6c, then node 6d will be the first node to receivethat packet (i.e., the first adjacent node in the direction of flowaround the ring is the first to receive the "empty" packet).

At node 6d, the "empty" packet is received by receiver 70a andeventually passed to packet handling circuit 78a. Packet handlingcircuit 78a receives circuit switched data which is read selectivelyfrom circuit switched data banks 122 and 126 in response to addressessupplied by map (ring 1) 96. In other words, by virtue of the addressesand control it supplies, ring map 96 causes particular bytes (orpossibly all of the bytes or none of the bytes) of "local" circuitswitched data stored in banks 122 and 126 to be selectively read fromthose banks and passed to the packet handling circuit 78a. A similarprocess occurs in parallel with map (ring 2) 98, memory 102 and packethandling circuit 78b.

Packet handling circuit 78a inserts the "local" circuit switched data itreceives (if any) into the payload of the received "empty" packet whilethat packet is passing to the transmitter 84a for transmission to thenext node on the inter-nodal network 12. This process is repeated suchthat each other node, in succession, has the opportunity to insert itsown "local" circuit switched data in the payload of the packet whichoriginated from node 6c. If a particular node has no "local" circuitswitched data to insert in the payload, the received packet passesunaltered to the next node. Eventually, the packet which was sent out"empty" traverses the entire ring on which it was transmitted andreturns "full" to the node from which it was transmitted (originated).At that node (6c), circuit switched data from the payload of the "full"packet is passed through ring select circuit 94, written sequentiallyinto receiver memory 108 and then time switched out as LSDATA or SLDATA.This method is referred to as the "Empty Send/Full Return" or ESFRmethod for shorthand.

The ESFR method is repeated such that each node, in turn, transmits an"empty" packet and receives a "full" return packet (on the node'sassigned ring), thereby enabling "local" circuit switched dataoriginating from any port at any node to be effectively transferred toany other port of the same or different node. All circuit switched datais preferably transferred in less than 125 μs to avoid loss of samples.As explained below, it should also be understood that the ESFR methodmay be used to "broadcast" or transfer information originating from oneport to more than one other port.

In the second method, the concept is for each node, in turn, tooriginate (transmit) a packet whose payload is "full" when sent, but"empty" upon return. Thus, a shorthand name for this method is the "FullSend/Empty Return" or FSER method. In the FSER method, all of the"local" circuit switched data stored in circuit switched data banks 122and 126 of transmitter memory 102 is read sequentially and supplied topacket handling circuit 78a. A "full" packet is constructed whosepayload includes all of the "local" circuit switched data for a givennode. The "full" packet is transmitted by transmitter 84a and isreceived by the first adjacent node. The data in the payload isselectively extracted and passed, via ring select circuit 94, toreceiver 68. That data is then selectively written into data banks 130and 134 of receiver memory 108. This process is repeated until a "full"packet transmitted by each node has been received by every other node,thus achieving the same overall result of enabling "local" circuitswitched data originating from any port at any node to be effectivelytransferred to any other port of the same or different node.

In addition to transferring circuit switched data between nodes,inter-nodal network 12 may also be used to transfer packet switcheddata. Examples of packet switched data are data or maintenanceinformation needed to control the switching system itself, X.25 packets,LAPB or LAPD packets. Packet switched data appears at the output of ringselect circuit 94, but is written into packet receiver memory 118, asopposed to memory 108. Once stored in memory 118, packet switched datais accessible by CPU 64 via CPU data bus 116.

The ESFR Method

Referring now to FIGS. 3A through 3E, 4A and 4B, further details of theESFR method will be described. It should be understood that theflowchart of FIG. 4B represents the steps which are performed, inparallel, at each node by that node's packet control circuits (92a and92b), the packet handling circuits 78a and 78b and related circuitry. Ifshould be kept in mind that when the ESFR method is used, "empty"packets are transmitted on only one ring and received on only one ring(assigned during initialization). For this example, it is assumed thatnode 6i in FIG. 4A is preparing to transmit an "empty" packet over theinter-nodal network 12 for the purpose of collecting circuit switcheddata from other nodes, including node 6j.

The process begins at start on reset step 138, which is a state in whichthe node is essentially waiting for a frame (which contains a packet) toarrive on the inter-nodal network 12. At step 140, a determination ismade whether the start of a frame has been detected. If a start of frameis not detected, the process returns to start 138. Alternatively, if thestart of a frame is detected, meaning that a packet was received by node6i, then the contents of the control portion of the packet are checkedto determine if the packet is "busy" at step 142. A packet's "busy" ornot busy ("free") status is indicated by the busy indicator (BI) in thecontrol portion of the packet (FIG. 1E). If the packet is not busy,meaning it is "free" for node 6i to use, the process proceeds to step144 where a determination is made whether the circuit switched data(CSD) window for node 6i is open. The "CSD window" refers to adesignated period of time which is allocated for all of the nodes totransmit "empty" circuit switched data packets.

If the CSD window is not open, meaning that it is not the appropriatetime for node 6i to transmit an "empty" packet for circuit switcheddata, then the process returns to start 138. If the CSD window is open,then the process advances to step 146 at which node 6i starts theprocess of sending a packet by transmitting a "busy" control word overthe network 12 to take control of the packet. Next, at step 150, node 6icontinues the process of sending an "empty" packet over the network 12.Note, however, that at step 148, node 6i must insert "local connectdata" (if any) into the payload of the "empty" packet while transmissioncontinues. The term "local connect data" refers to circuit switched datawhich is both originating from and destined for one or more local portsof a given node which is sending an "empty" packet. In other words,local connect data is circuit switched data which is to be switched fromone local port to another local port of the same node over inter-nodalnetwork 12. Thus, in this example, if node 6i has any local ports whichare connected to each other, the circuit switched data pertaining tothose ports would be inserted into the payload of the "empty" packet atstep 148. In effect, node 6i (or any other node) transmits local connectdata to itself. Next, at step 152, the transmit flag (XF) 90a (FIG. 3A)is set to serve as a reminder to node 6i that it has transmitted an"empty" packet over the network 12 and that it should receive the return"full" packet in the future.

Next, the process returns to start 138 to await receipt of anotherframe. Once the start of another frame is detected and it is determinedthat the packet within the frame is "busy" (not free), the processadvances to step 154 where a determination is made as to whether thetransmit flag is set. If XF is not set, meaning that the packet whichwas just received originated from another node, then the processproceeds to step 162 where address information contained in the controlportion of the packet is checked to determine the (nodal) source of thepacket. Thus, in this example, when node 6j actually receives the"empty" packet transmitted by node 6i, the process would advance to step162 because node 6j's transmit flag would not be set. At this point,node 6j must insert appropriate circuit switched data into the payloadof the packet. In this example, the appropriate circuit switched data isdata pertaining to any of node 6j's local ports which already are (orare about to be) connected to any of node 6i's local ports. As shown inFIG. 4A, this is accomplished by CPU 64a in node 6j writing address andcontrol data into one of the address maps 96,98 such that theappropriate circuit switched data is written selectively into thepayload of the received packet at step 164. This step represents thebeginning of a second stage of switching (node to node) performed by thesystem 17. Error status information is then placed in the status andcontrol portion of the packet at step 165.

Next, under normal circumstances, the now "full" return packet isreceived by node 6i. If so, the process advances through steps 138, 140and 142, to step 154 where again a determination is made (this time bynode 6i) as to the status of the transmit flag. Since node 6i previouslyset its transmit flag (at step 152 when the "empty" packet wastransmitted), that node determines that the flag is indeed set. At step156, the busy indicator in the control portion of the packet is changedso that the packet, when passed to the next node, is "free" and may beused by another node. The circuit switched data contained in thepayload, which consists of any local connect data that was inserted atstep 148 along with all circuit switched data inserted by each othernode (including node 6j), is then written sequentially into the receivermemory 108. Finally, the transmit flag is reared at step 160 and errorstatus information is checked at step 161 before the process returns tostart 138. When circuit switched data is eventually time switched out ofmemory 108, it is processed by pad lookup circuit 110 which operates ina conventional manner to perform A-law to μ-law (or vice versa)conversions.

FIGS. 4C and 4D show an embodiment of the ESFR method in which bothcircuit switched data and packet switched data may be transferredbetween nodes. The initial steps are the same as those shown in FIG. 4B.However, note at step 144 that when a particular node determines thatthe CSD window is not open, meaning that its circuit switched data wasalready transmitted (in the current 125 μs frame), the process advancesto step 155 instead of returning immediately to start 138. At step 155,a determination is made whether an "empty" data packet, which will beused to collect packet switching information from other nodes, is readyfor transmission and the receiver memory is ready. If the "empty" datapacket is not ready or the receiver memory is full (not ready), theprocess returns to start 138. Otherwise, the process advances to step157 at which information in the control portion of that packet ischanged to designate the packet as "empty". The "empty" packet is thentransmitted at step 159, the transmit flag is set at step 161, and theprocess returns to start 138.

When the next frame is received, the process advances through steps 138,140 and 142. Assuming that the received packet (within the frame) isdesignated "busy," the process advances to step 154 where the status ofthe transmit flag is checked. If the transmit flag is set, meaning thatthe node receiving this packet previously transmitted either an "empty"packet to collect packet switched data (at steps 159, 161) or an "empty"packet to collect circuit switched data (at steps 148-152), then theprocess advances to step 166 where a determination is made of what typeof packet has just been received, again by examining information in thecontrol portion of the packet. The type of packet is indicative ofwhether the packet's payload contains circuit switched data, packetswitched data or possibly other types of data (e.g., voice processing ormaintenance). If the packet is the type that carries circuit switcheddata, the process advances through steps 158 and 160, just as describedin connection with FIG. 4B. If the packet is the type that carriespacket switched data, the process advances to step 168 where adetermination is made whether the packet is full. If the packet is notfull, it means that no other node had any packet switched data to send(at least during the period of time it took for the packet to traversethe network) to the node which originally transmitted (and has justreceived) that packet. In that event, the transmit flag is reared atstep 171 and the process returns to start 138.

On the other hand, if it is determined that the packet is full at step168, then the process advances to step 170 where a buffer counter isincremented. Next, the packet is copied into the data packet receivermemory 118 (FIG. 3C) where it is temporarily stored awaiting furtherprocessing. The transmit flag is then cleared at step 174. Lastly, theCPU 64b is notified of the arrival of a packet switched data packet byan interrupt at step 176.

With reference again to step 154, if a determination is made that thetransmit flag is not set, meaning that the packet which was justreceived originated from another node, then the process advances to step182 where, like step 166, a determination is made regarding the packettype. If the packet is of the type that carries circuit switched data,the process proceeds through steps 162, 164 and 165, just as in FIG. 4B.If the packet is the type that carries packet switched data, then theprocess advances to step 188 where a determination is made whether thepacket is "empty." If the packet is not "empty," meaning that anothernode already filled the payload, the packet passes to the next node andthe process returns to start 138.

Alternatively, if the packet is "empty," meaning it was originallytransmitted "empty" by another node for the purpose of collecting packetswitched data and no other node has already "filled" the payload, thenthe process advances to step 190 where the node which has received thepacket determines whether it has any packet switched data to send to thenode which originally transmitted the packet. If not, the "empty" packetis passed to the next node and the process returns to start 138. If so,the packet is marked "full" at step 192, the packet switched data isplaced in the payload and the "full" packet is transmitted to the nextnode at step 194.

FIG. 4E is a timing diagram showing a preferred embodiment forallocating the bandwidth of the inter-nodal network 12 to allowtransfers of both circuit switched data and packet switched data by allnodes. In this embodiment, transfers of data over the inter-nodalnetwork are made within framing windows, each of which is 125 μs induration. A period of 125 μs is preferred since it corresponds with thesampling rate (8 kHz) of most widely used network protocols, meaningthat the values of circuit switched data may change every 125 μs.Therefore, by requiring that all inter-nodal transfers of circuitswitched data take place in less than 125 μs, inter-nodal network 12ensures that all such data is transferred before any values change. Thisalso permits the inter-nodal network 12 to operate asynchronously withrespect to the PSTN (or private network) 10.

Within each framing window, approximately one-half of the available time(i.e., 62.5 μs) is allocated for all nodes, in round-robin fashion, totransfer circuit switched data to other nodes. Such transfers may bemade using either the ESFR or FSER method, or both, and may involve anytype of packet carrying packet switched data (or even circuit switcheddata which is being used for another purpose), including packets 5, 7and 9 of FIG. 1E. The remaining time within each window is allocated fornodes to transfer packet switched data (if any) to other nodes. Notethat "priority" is given to the circuit switched data, since all suchdata from all nodes is transferred before any packet switched data maybe transferred.

The ESFR method may also be used to "broadcast" circuit switched data tomultiple ports of the same node or across multiple nodes. For example,if there is "local" circuit switched data which is intended forbroadcast to multiple local ports, multiple copies of that data issimply inserted into the payload of the "empty" packet at step 148(FIGS. 4B and 4C). In other words, multiple copies of the byte of datathat is intended for broadcast are selectively placed in the payload inlocations corresponding to the local ports which are to receive thebroadcast. Similarly, if circuit switched data from a remote port isintended for broadcast, multiple copies of that data are inserted atstep 164 into locations in the payload(s) (i.e., one packet/payload isneeded for each node which has a port that is supposed to receive thebroadcast) corresponding to the intended ports.

To summarize, as reflected in FIG. 4A, when the ESFR method is used totransfer data, each node in round-robin fashion transmits an "empty"packet for the purpose of collecting data from all other nodes served bythe inter-nodal network 12. Upon receipt of an "empty" packettransmitted by another node, each node operates to selectively read datafrom one of its memories and place it in the payload of the "empty"packet. When the now "full" packet eventually returns to the node whichtransmitted it, the data contained within the payload is sequentiallywritten into one of that node's receiver memories. This step marks thecompletion of the second stage of switching (one-way node to node)performed by the system.

The FSER Method and Combined ESFR/FSER Method

With reference to FIGS. 5A through 5C, further details of the FSERmethod will be described in the context of a preferred embodiment of a"combined" method in which the FSER method is used to transfer packetswitched data and the EFSR method is used to transfer circuit switcheddata. For enhanced clarity, the portions of FIGS. 5B and 5C whichrepresent the FSER method are enclosed with broken lines. The portionsof FIGS. 5B and 5C which represent the EFSR method lie outside of thebroken lines and are identical to the steps of FIGS. 4C and 4D which aredenoted by like reference numbers.

At step 144, if a determination is made that the CSD window is not open,meaning that it is not the appropriate time to collect circuit switcheddata from other nodes, the process advances to step 196 where adetermination is made whether a "full" data packet (containing packetswitched data) is ready for transmission to another node. If not, theprocess returns to start 138 to await the arrival of another frame. If adata packet is ready, meaning that the payload of the packet is loadedwith the packet switched data and an appropriate (nodal) destinationaddress is placed in the control portion of the packet, the packet ismarked "full" at step 198. The "full" data packet is then transmitted atstep 200. Next, the transmit flag is set at step 202 and the processreturns to start 138 to await the arrival of another frame.

Now, consider what happens when a "full" data packet which wastransmitted by one node is received by another node. The processadvances through steps 138, 140 and 142 to step 154 where adetermination is made as to whether the receiving node's transmit flagis set. If that flag is not set, meaning that the packet originated froma different node, the process advances to step 182 where it isdetermined, in this example, that the packet contains packet switcheddata as opposed to circuit switched data. Next, at step 214, the nodaldestination address of the packet is checked to determine whether thereceiving node is the intended recipient of the packet. If not, theprocess returns to start 138. If so, the receiving node checks to see ifits packet receiver memory 118 (FIG. 3A) is ready to accept the packetat step 216. If memory 118 is not ready to accept (e.g., because thememory is currently full), the process advances to step 220 whereinformation is inserted into the status and control portion of thepacket to indicate that the node was busy and was unable to accept thepacket. The process then returns to start 138.

Alternatively, at step 216, if memory 118 is ready to accept the packet,the process advances to step 218 where the packet is copied into thatmemory. Next, at step 222, the CPU 64b is notified of the arrival of apacket switched data packet by an interrupt.

Lastly, we shall consider the situation where a "full" data packetreturns to the node which transmitted it. In this instance, the processadvances from step 138 to step 154 where it is determined that thereceiving node's transmit flag is indeed set. At step 156, the packet'sbusy indicator is released (changed to "free") followed by adetermination at step 166 of what type of data the packet contains. Inthis example, the packet contains packet switched data, so the processadvances to step 204 where the transmit flag is cleared. Next, at step206 a determination is made, based on information contained within thestatus and control portion of the packet, as to whether the node to whomthe packet was addressed was busy. If so, meaning the packet was notaccepted by the destination node, the process returns to start 138 tomake another attempt to deliver the packet to its destination. If not,the packet transmitter memory (constant areas 124 and 128 in FIG. 3F) ismarked empty at step 208. A determination is then made at step 210whether the packet was accepted by the destination node to which it wasaddressed. If so, the process returns to start 138. If not, errors arelogged at step 212 before returning to start 138.

It should be apparent that the FSER method may be used to transfercircuit switched data as well as packet switched data. When circuitswitched data is to be transferred, each node, in turn, transmits a"full" packet whose payload is filled with circuit switched data (forall local ports) which is read sequentially from the transmitter memory102. As a given node receives, in turn, a "full" packet transmitted byevery other node, the given node takes appropriate data from the payloadof each such packet and selectively writes data into its receiver memory108 in response to addresses supplied by sequential counter/map 104.Note that the addresses supplied by counter/map 104 are "global"addresses (i.e., the combination of the implicit port address and thenodal source address), meaning each may represent any port of any nodein the entire system. Because the circuit switched data corresponding tothese global addresses is written to locations in memory 108 (whichcorrespond to local ports), an address translation must be performed inorder to eventually read such data out of memory 108 in the correctorder. An address map translation circuit 105 receives as inputs theaddresses produced by sequential counter/map 104 of memory 108 wheredata is stored. The addresses produced by address map local 107 are usedto select constant areas within memory 108 and pad values from padlookup 110.

Like the ESFR method, the FSER method may be used to broadcast circuitswitched data to multiple ports. At a given single node, this isaccomplished by making multiple copies of the data intended forbroadcast from the payload of a "full" packet and selectively writingsuch data into multiple locations of that node's receiver memory.Similarly, different nodes may be instructed to copy the same broadcastdata from the payload of a "full" packet and selectively write such datainto one or more locations of those nodes' respective receiver memories,thereby effecting broadcasting across multiple nodes.

Connecting Calls Between Nodes

Having presented various alternatives for transferring informationacross inter-nodal network 12, a specific example of how a call isconnected between ports which are physically associated with differentnodes will now be described. With reference once again to FIGS. 1A, 1B,2A and 3A through 3E, it should be kept in mind that each node 6a-6hnecessarily includes at least one nodal switch 44a. We shall assume thata calling party whose line is interfaced with node 6h goes off-hook anddials a number which corresponds to a called party whose line isinterfaced with node 6e. The host 4 receives a "request for service"message (which may include the dialed digits) from CPU 64 in node 6h.The host 4 determines that a connection must be established betweennodes 6h and 6e and, in response, issues a "connect" message (with portaddress information) to both nodes' CPUs 64 to connect to each other.

Now, let us consider for a moment what happens just at node 6h. Circuitswitched data from the calling party's line is initially passed, via bus30a, from one of the line cards 20 to nodal switch 44a. For purposes ofthis example, we shall further assume that that data is stored intransmitter memory 102. Next, if the ESFR method is used, when an"empty" packet transmitted (originated) by node 6e over the inter-nodalnetwork 12 is received by node 6h, the circuit switched data from thecalling party is time switched out of memory 102 and inserted into thepayload of that packet, which will eventually return to node 6e. At thispoint, a one-way circuit switched connection exists between the callingparty (node 6h) and node 6e, a "time" portion executed by thetransmitter memory 102 and a second stage portion executed by theinter-nodal network 12. Next, node 6e's receiver 68 receives its return"full" packet containing the circuit switched data from the callingparty. That data is time switched through receiver memory 108 and passedvia bus 30a to the line card 20 to which the called party is interfaced.At this point, a complete one-way connection exists between the callingparty (node 6h) and the called party (node 6e). Exactly the same processis repeated, in reverse, to establish the other half of the desiredtwo-way connection.

Alternatively, the FSER method could be used to connect the same call.In that case, transmitter 102 in node 6h time switches the callingparty's circuit switched data into a "full" packet which is transmittedover the inter-nodal network 12. Node 6e, upon receipt of the "full"packet, extracts the calling party's circuit switched data, stores thedata in receiver memory 108, and time switches the data to the line cardto which the called party is interfaced. Again, the process is carriedout in reverse to establish the other half of a two-way connection.

FIGS. 6A and 6B show the expandable telecommunications system 17 (FIGS.1C and 1D) modified to illustrate the effect of a failure ofprogrammable switching node or a portion of the inter-nodal network 12.In this example, node 6f has failed or a portion of inter-nodal network12 has failed (or possibly a malfunction was detected and the node wastaken out of service by the host 4). The nodes 6e and 6g which areadjacent to the failed node 6f begin to operate in "loopback" mode. Inloopback mode, the circuitry within a node which is normally used toreceive information from one ring is connected to the circuitry which isnormally used to transmit information on the other ring, as denoted byreference numbers 71a and 71b in both FIGS. 3A, 6A and 6B. Thus, when agiven node operates in loop back mode, all information received on onering is immediately transmitted on the other ring. A particular node maybe instructed by the host 4 to operate in loopback mode or,alternatively, operation may begin automatically in response toexpiration of a "watchdog" timer.

By virtue of the loopback mode and the fact that two rings instead ofone are used to form the inter-nodal network 12, the fault created bythe failure of node 6f is effectively isolated from the rest of thesystem 17. That is, only the local ports of node 6f suffer a loss ofservice due to the failure of that node.

FIG. 7 shows another alternative embodiment of the present invention inwhich four programmable switching nodes 6k-6n are connected together byan inter-nodal network 12 which consists of one pair of rings, pair A,and one redundant pair of rings, pair B. It should be understood thatthis embodiment is not limited to only four switching nodes and that oneor more additional nodes may be added. In this embodiment, the bandwidthof pair A is preferably sufficiently large that under normal operatingconditions, all data (i.e., circuit switched and packet switched) may betransferred by that pair alone. Pair B preferably has comparablebandwidth to that of pair A and remains in a "standby" mode under normalconditions. In the event of a failure of either of pair A's rings, pairB enters a regular operating mode and assumes responsibility fortransferring all of the data. Also, it is preferable that only one pairof rings is "active," but that both pairs actually transfer informationbetween nodes in parallel. This is to ensure that, in the event of afailure of the "active" ring, connections (calls) which are alreadyestablished can be maintained and not dropped.

FIGS. 8A and 8B depict another alternative embodiment of the presentinvention in which a two-ring inter-nodal network 12 is used to connecta plurality of voice processing resources 224a-224e with a plurality ofprogrammable switching nodes 6p and 6q to provide a voice processingsystem 226. (A single ring network could also be used). Voice processingresources 224a-224e may represent the same or different call processingor communications services including voice mail, interactive voiceresponse, fax mail, voice messaging or other enhanced services or dataprocessing services. Because voice processing resources 224a-224e do notinclude any network/line interfaces (and therefore require no framinginformation), those resources may advantageously operate asynchronouslywith respect to the PSTN (or private network) 10. In addition, resources224a-224e may be configured to appear as servers with respect to eachclient host 4.

FIG. 8C shows a preferred embodiment of voice processing resource 224a.Note that the components of resource 224a are essentially the same asthose of the switching node 6 shown in FIGS. 2B and 2C, except thatresource 224a does not require and does not have any line cards or othercards (i.e., MFDSP and ISDN-24) normally needed for network/lineinterfaces.

All voice processing resources 224a-224e preferably appear as nodes onthe inter-nodal network 12 and have the same access to the bandwidth asother (switching) nodes. Such access is highly advantageous because itpermits any resource 224a-224e to dynamically provide desired servicesto any port served by the system 226. For example, assume that a calleron a local port of node 6q wishes to access a voice mail system toeither leave a message for someone who did not answer or to retrievemessages. Using either the ESFR or FSER method, the caller may beconnected with any of voice processing resources 224a-224e. Assumingthat one of those resources is a voice mail system, the caller isprovided with the desired service. Of course, the caller may likewise beconnected to any of the other voice processing resources which areserved by the inter-nodal network 12.

FIGS. 9A and 9B show yet another embodiment of the present invention inwhich multiple inter-nodal networks are connected together to form asystem 228 having even greater switching capacity or combinedswitching/voice processing capacity. A first two-ring inter-nodalnetwork 12c (which provides switching capacity through programmableswitching nodes 6r and 6s), is connected to a second two-ringinter-nodal network 12d (which provides voice processing capacitythrough nodes 224f-224i and switching capacity through node 6t) by aprogrammable switching node bridge 230. For purposes of enhancedclarity, an additional pair of redundant rings for each of networks 12cand 12d is omitted from this figure.

Bridge 230 appears as a node on both inter-nodal networks 12c and 12dand is therefore interfaced with each of rings 1, 2, 5 and 6. By virtueof its access to both inter-nodal networks, bridge 230 is operable toexchange information bidirectionally between networks 12c and 12d. Forexample, bridge 230 may effectively connect any local port of node 6r or6s (or any other node of network 12c) to any voice processing resource224f-224i or local port of switching node 6t of network 12d. Inter-nodalnetworks 12c and 12d may operate at different speeds without adverselyaffecting bridge 230.

As shown in FIG. 9C, bridge 230 includes essentially the same componentsas a programmable switching node, but also includes two additional ringIO cards 40c and 40d, and two additional nodal switches 44e and 44dwhich permit bridge 230 to interface with two additional inter-nodalnetworks 12e and 12f. Although only two additional nodal switches 44cand 44d are shown, it is possible to add even more such switches, all ofwhich will cooperate in the manner about to be described. Also, bridge230 does not require any network/line interfaces (or associated IO cardsand line cards), although it may optionally include such components.

With reference now to both FIGS. 3A through 3E and 9C, an example of howinformation may be transferred between inter-nodal networks 12c and 12dwill be described. First, one should understand that FIGS. 3A through 3Eillustrate the basic hardware each of nodal switches 44a-44d of bridge230. That is, each nodal switch 44a-44d is essentially a replica of theswitch disclosed in FIGS. 3A through 3E. Nodal switch 44a is configuredas the local bus master (active), and nodal switch 44b is configured asa redundant local bus master. Nodal switch 44c is configured as astandard nodal switch (active), and nodal switch 44d is configured as aredundant standard nodal switch.

It may be recalled that each nodal switch 44a-44d includes a transmittermemory 102 which is operable for storing circuit switched data that isflowing in a direction from a line card to a switch (LSDATA) or,alternately, from a switch to a line card (SLDATA). Similarly, eachswitch's receiver memory 108 is operable for outputting either LSDATA orSLDATA. As there are no line cards included in bridge 230 (althoughthere may be such cards), it may be conceptually helpful to think ofLSDATA as circuit switched data which is flowing in a direction fromnodal switch 44c (and 44d) to nodal switch 44a (and 44b), and to thinkof SLDATA as circuit switched data which is flowing in a direction fromnodal switch 44a (and 44b) to nodal switch 44c (and 44d). For purposesof this discussion, it is assumed that nodal switches 44a and 44b areactually configured to accept and store LSDATA in their transmittermemories 102 and to output SLDATA from their receiver memories 108. Itis further assumed that nodal switches 44c and 44d are configured toaccept and store SLDATA in their transmitter memories 102 and to outputLSDATA from their receiver memories 108.

The objective of this arrangement is that whatever circuit switched data(including data received from inter-nodal network 12c) that is timeswitched through nodal switch 44a (or 44b, if it becomes active) ispassed to nodal switch 44c (and 44d). In turn, nodal switch 44c isoperable to transfer data it receives from switch 44a onto inter-nodalnetwork 12d. The converse is also true, meaning that all circuitswitched data (including data received from inter-nodal network 12d)that is time switched through nodal switch 44c (or 44d, if it becomesactive) is supplied to nodal switch 44a (and 44b), from which such dataor portions thereof may be transferred over inter-nodal network 12c.Thus, the combined effect of this arrangement is that circuit switcheddata which originates from any node on either inter-nodal network 12c or12d may be transferred to any other node on either network. Packetswitched data is transferred by bridge 230 from nodal switch to nodalswitch across the bridge's HDLC bus.

In terms of implementing desired redundancy features, communicationsservices provided by MFDSP cards 36 and ISDN-24 cards 38 as well as evenfurther expansion of the telecommunications system, space switch controlcircuit 112 (FIG. 3C) is instrumental. The function of circuit 112 is topermit, on a time slot-by-time slot basis, one and only one device ofall of the nodal switches 44, MFDSP cards 36 and IDSN-24 cards 38 totransmit circuit switched data over bus 30a. In terms of redundancyfeatures, circuit 112 has the following effect. When nodal switch 44a isactive and functioning properly, circuit 112 within redundant switch 44bwill effectively prevent switch 44b from transmitting any circuitswitched data over bus 30a, although 44a is permitted to receive alldata passing over that bus. Should nodal switch 44a fail, then circuit112 would permit redundant switch 44b to commence transmitting data overbus 30a during those time slots in which switch 44a, if functioningproperly, would normally transmit. The same considerations apply toswitch 44c and its redundant pair switch 44d.

In terms of communications services, circuit 112 operates to dynamicallyprevent nodal switches 44a and 44c from effectively transmitting circuitswitched data over bus 30a during time slots in which a service is beingprovided by any of cards 36 or 38. Details of how "ownership" or theauthority to transmit data during a given time slot may be dynamicallypassed from one device to another (and back again) are disclosed inco-pending application Ser. No. 08/001,113, incorporated by referenceabove.

The role of circuit 112 in connection with even further expansion of atelecommunications system is described in connection with FIGS. 10B and10C.

FIG. 10A shows another embodiment of the present invention in which upto sixteen programmable switching nodes 234 are connected together byfour inter-nodal networks 12g-12j (a total of eight rings) to form anexpanded telecommunications switching system 232. Although only sixteennodes are illustrated, it should be understood that the number of nodesmay be greater depending upon the switching capacity of each node andthe rate at which information may be transferred over the inter-nodalnetworks 12g-12j. It should also be apparent that even further expansionof the switching capacity of system 232 may be achieved by addingadditional inter-nodal networks.

Under normal operating conditions, inter-nodal networks 12g and 12i arepreferably active and are used to transfer all information between allnodes. The remaining inter-nodal networks 12h and 12j preferably havecomparable bandwidth to that of 12g and 12i and transfer information isparallel with 12h and 12j, but remain in a "standby" mode. In the eventof a failure of either of the rings of networks 12g and 12i, thecorresponding redundant network becomes active.

FIGS. 10B and 10C show the major components of one of nodes 234. Thecomponents and their operation are comparable to those discussed earlierin connection with other figures. Note that by adding additional ring IOcards 40 and nodal switches 44, additional inter-nodal networks 12k maybe added to system 234, thereby even further expanding the switchingcapacity of system 232.

As mentioned above, space switch control circuit 112 (FIG. 3C) plays arole in system 234. Circuit 112's function is to ensure, on a timeslot-by-time slot basis, that one and only one of the multiple,non-redundant nodal switches 44a, 44c and 44d present (as well as anyMFDSP cards 36 and ISDN-24 cards 38 which are present) effectivelytransmits circuit switched data over bus 30a. Thus, control circuit 112enables multiple nodal switches (even beyond those shown) to be added toa node, even further increasing the overall switching capacity of thesystem.

The foregoing description has been limited to a specific embodiment ofthis invention. It will be apparent, however, that variations andmodifications may be made to the invention, with the attainment of someor all of the advantages of the invention. Therefore, it is the objectof the appended claims to cover all such variations and modifications ascome within the true spirit and scope of the invention.

What is claimed as new and desired to be secured by Letters Patent ofthe United States is:
 1. A method of transferring information between aplurality of nodes which are connected in communicating relationship byan inter-nodal network to form an expandable telecommunications system,said method comprising the steps of:(a) transmitting a packet over saidinter-nodal network from one of said nodes when said packet is notutilized by another node, said packet containing information identifyingsaid transmitting node, wherein said packet either contains informationwhich said transmitting node intends to transfer to one or more othernodes or having a capacity to carry information; (b) receiving at one ofsaid one or more other nodes said packet; (c) determining if saidreceived packet is an empty packet or full packet; (d) transmittinginformation destined for said transmitting node via said packet fromsaid receiving node when said received packet is an empty packet; (e)receiving said information destined for said receiving node at saidreceiving node when said received packet is full; and (f) repeating step(a) through (e) at each of said one or more other nodes.
 2. The methodas in claim 1, wherein said packet comprises:a source address configuredto store said information identifying said transmitting node; and apayload for containing said information.
 3. The method as in claim 2wherein during said step (a), said transmitting node performs the stepsof:(1) sequentially reading circuit switched data from a localtransmitter memory; and (2) inserting said sequentially read data intosaid payload, said sequentially read data originating from one or moreports which are physically associated with said transmitting node. 4.The method as in claim 3, wherein during said step (e), said receivingnode performs the step of:(1) selectively retrieving from the packet anyinformation which is destined for said receiving node; and (2)selectively writing circuit switched data from said payload into a localreceiver memory, said selectively written data destined for one or moreports which are physically associated with said receiving node.
 5. Themethod as in claim 3, wherein during said step (a), said transmittingnode sequentially performs the steps of:(3) reading packet switched datafrom a local transmitter memory; (4) inserting said sequentially readdata into said payload; and (5) inserting into said packet a destinationaddress which identifies one or more of said other nodes as the intendeddestination or destinations of said packet switched data.
 6. The methodas in claim 5, wherein said circuit switched data selectively insertedinto said payload at said receiving node during said step (f) comprisescircuit switched data which is destined for one or more ports physicallyassociated with said transmitting node.
 7. The method as in claim 6,further comprising the step of:(i) sequentially writing the circuitswitched data retrieved from the payload into a local receiver memory atthe transmitting node when a transmitted empty packet is returned tosaid transmitting node.
 8. The method of claim 2, wherein said step (f)comprises the steps of:(1) selectively inserting information destinedfor said transmitting node into said packet; and (2) after said step(f)(1), allowing said packet to continue along said inter-nodal networkto another node.
 9. A method of transferring information between aplurality of nodes which are connected in communicating relationship byan inter-nodal network to form an expandable telecommunications system,said method comprising the steps of:(a) transmitting a packet over saidinter-nodal network from one of said nodes, said packet containingsource address information and information, if any, which saidtransmitting node intends to transfer to each of the remaining nodes;(b) receiving said packet at a first one of said remaining nodes andselectively extracting any information destined for such receiving node;(c) repeating step (b) at each of said remaining nodes, in turn, untilsaid packet returns to the transmitting node; and (d) repeating steps(a) through (c) until each of said nodes has received a packet fromevery other node.
 10. The method as in claim 9 wherein during said step(a), said transmitting node performs the steps of:(1) sequentiallyreading circuit switched data from a local transmitter memory; and (2)inserting said sequentially read data into a payload within said packet,said sequentially read data originating from one or more ports which arephysically associated with said transmitting node.
 11. The method as inclaim 10 wherein said circuit switched data is arranged in apredetermined order within said payload, whereby said receiving node mayuse said order to determine the ports from which said circuit switcheddata originated.
 12. The method as in claim 9 wherein during step (b),said receiving node performs the step of:(1) selectively writing circuitswitched data from said payload into a local receiver memory, saidselectively written data destined for one or more ports which arephysically associated with said receiving node.
 13. The method as inclaim 9 wherein during said step (a), said transmitting node performsthe steps of:(1) reading packet switched data from a local transmittermemory; (2) inserting said read data into a payload within said packet;and (3) inserting into said packet a destination address whichidentifies one or more of said remaining nodes as the intendeddestination or destinations of said packet switched data.
 14. The methodas in claim 13 wherein during said step (b), said receiving nodeperforms the steps of:(1) checking said destination address of saidpacket; and (2) writing said packet switched data contained in saidpayload into a local packet receiver memory when said destinationaddress indicates said packet is destined for said receiving node. 15.The method as in claim 13 wherein during said step (b), the receivingnode performs the steps of:(1) verifying that the destination addressindicates that said received packet is destined for said receiving node;(2) determining if a local packet receiver memory is ready to acceptsaid packet switched data; and (3) writing said packet switched datacontained in said payload into said local packet receiver memory whensaid local packet receiver memory is ready.
 16. The method as in claim13 wherein during said step (b), the receiving node performs the stepof:(1) verifying that the destination address indicates that saidreceived packet is destined for said receiving node; (2) determining ifa local packet receiver memory is ready to accept said packet switcheddata; and (3) forwarding said packet to said transmitting node when saidlocal packet receiver memory is not ready.
 17. The method of claim 9,wherein said steps (a) through (d) are completed within 125microseconds.
 18. A method of transferring information between aplurality of nodes which are connected in communicating relationship byan inter-nodal network to form an expandable telecommunications system,said method comprising the steps of:(a) transmitting a packet over saidinter-nodal network from one of said nodes, said packet containinginformation identifying said transmitting node and having a capacity tocarry information; (b) receiving said packet at a node other than saidtransmitting node, said receiving node performing the steps of:(1)selectively inserting information destined for said transmitting nodeinto said packet, and (2) after said step (b)(1), allowing said packetto continue along said inter-nodal network to another node; (c)repeating step (b) at each node other than said transmitting node, inturn, until said packet returns to said transmitting node; (d) at saidtransmitting node retrieving from said returned packet the informationinserted therein by other nodes; and (e) repeating steps (a) through (d)until each of the plurality of nodes has transmitted a packet andreceived that returned packet containing information from other nodes.19. The method as in claim 18 wherein said packet includes a sourceaddress which identifies said transmitting node and a payload forcarrying said information.
 20. The method as in claim 19 wherein duringsaid step (a), said transmitting node performs the steps of:(1)selectively reading circuit switched data from a local transmittermemory; (2) inserting said selectively read data into said payload, saidselectively read data both originating from and being destined for oneor more ports that are physically associated with said transmittingnode.
 21. The method as in claim 20 wherein said circuit switched dataselectively inserted into said payload at said receiving node duringsaid step (b) comprises circuit switched data which is destined for oneor more ports physically associated with said transmitting node.
 22. Themethod as in claim 21 wherein said circuit switched data is arranged ina predetermined order within said payload, whereby said receiving nodemay use said order to determine the ports from which said circuitswitched data originated.
 23. The method as in claim 20 wherein saidstep (d) includes the step of:(1) sequentially writing the circuitswitched data retrieved from the payload into a local receiver memory.24. The method of claim 23, wherein during said step (d) saidtransmitting node further comprises the following steps before said step(d)(1):(2) determining that said received packet originated with saidtransmitting node.
 25. The method as in claim 19 wherein during saidstep (b)(1), said receiving node performs the steps of:a) prior to saidstep (b), verifying that packet switched data destined for thetransmitting node is ready for transmission; and b) inserting saidpacket switched data into said payload when said packet switched data isready for transmission.
 26. The method as in claim 25 wherein duringsaid step (d), the transmitting node performs the step of:(1) writingthe packet switched data contained in said payload into a local packetreceiver memory.
 27. The method of claim 20 wherein during said step (a)said transmitting node further performs the following step before saidstep (a)(1):(3) taking control over said packet when said packet is notutilized by another node and when said transmitting node can transmitsaid packet during a period of time allocated for nodes to transmitempty circuit switched data packets.
 28. The method of claim 21 whereinsaid step (a)(1) comprises the step of:(i) writing address and controldata from a processor into an address map of said receiving node suchthat appropriate circuit switched data is selectively inserted into saidpayload.
 29. The method of claim 18 wherein said steps (a) through (d)are completed within 125 microseconds.